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CONTRACTOR REPORT 


ELEMENTS OF ENGINEERING EXCELLENCE 


Introduction 


The inspiration for this paper originated in discussions with the director of MSFC 
Engineering in 2006 who asked that we investigate the question: “How do you achieve 
excellence in aerospace engineering?” The authors’ approach to answering this question 
was a short course on Excellence in Engineering which is documented in this report. It has 
been taught as an adjunct to the Lessons Learned course at MSFC. The approach, a total 
system approach, forms a triad consisting of: Technical Understanding and Execution, 
Partnership With the Project, and Individual and Organizational Culture. We first looked at the 
“Root Causes” of various failures that NASA has had in its major projects. We found five 
fundamental root causes, which served as a basis for developing the above elements and 
sub-elements of the process. The application of these elements produces a path that leads to 
excellence in the product. We will discuss each of the elements and its sub elements and 
conclude with a short discussion of a Learning Organization as a key element in transforming 
the culture to achieve “Engineering Excellence.” 

There are a number of examples and figures cited in this report that are extracted from 
the Lessons Learned in Engineering report, NASA CR-201 1-216468, and in the Engineering 
the System and Technical Integration report, NASA CR-201 1-216472. 


Principles of Engineering Excellence 

o Overview 

o Problem Root Causes 
o Solutions 


The approach used to derive the principles of engineering excellence is shown on 
Figure 1. The process began with a study of the major incidents experienced by the authors 
working for NASA. This study was used to develop root causes from technical, 
organizational and cultural considerations. A path to excellence was developed based on 
Technical Understanding and Execution, Partnership with the Project, and Individual and 
Organizational Culture. 



Expanding this triad: Technical Understanding and Execution addresses 
understanding the physics, ensuring integration, interactions and interfaces, and sensitivities, 
uncertainties, and margins. Partnership with the Project addresses technical authority, 
requirements management, risk management, and cooperative solutions. Individual and 
Organizational Culture addresses ownership and accountability, critical thinking vs. 
procedures, right people in right places, and the learning organization. 


Process for Excellence 



Figure 1. Process for Achieving Engineering Excellence 


Root Causes of NASA Failures/Problems 

This study identified five top root causes (others can be added by the reader) that 
have led to major problems in NASA projects. 

1 . Shifting from engineering “hands-on” and “excellence” to “insight/oversight”. Lack of 
ownership. 

2. “Normalization of the deviances”. Not questioning anomalies. 

3. Lack of critical thinking. Over-reliance on procedures and computer codes. 

4. Decentralization of authority. 

5. Organizational and technical complexity. 
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The five root causes are not listed in any priority order. The first root cause listed 
deals with a shift in the NASA culture, where the organization moved from a hands-on 
engineering approach to an insight/oversight approach. In the early days the heritage was 
basically the arsenal approach where you designed, built and verified the system before 
contracting it out for production. The engineers really understood the design, the 
hardware/software and the system based on actual experience. In the early culture much of 
the technology development was an in-house, hands-on activity. The shift has resulted in the 
elimination of much independent analysis and test and experience based understanding of 
the systems required to catch and prevent problems. Howard E. McCurdy in his book Inside 
NASA says, “NASA officials from the original cultures believed they needed to provide their 
engineers and scientists with hands-on experience in order to maintain the technical side of 
the house. It was the only way to keep them technically sharp. By keeping their own 
engineers and scientists sharp, they could penetrate the work of the contractor. ... During the 
first decade of space flight, a strong technical culture guided the work of NASA employees. 
The norms typical of that period required NASA to maintain a corps of professional 
employees deeply involved in the details of space flight and aeronautics. The technical 
culture counterbalanced many organizational forces that rose up to challenge it. It 
overpowered the usual bureaucratic tendencies present in government operations. It 
provided a counterweight to the centralizing and organizational necessities of the Apollo 
mission.” [McCurdy, H. 1993] The loss of the technical excellence based on hands-on 
experience has led to many of the problems and therefore is one of the root causes of 
problems. To prevent problems, NASA needs to re-establish the culture of technical 
excellence based on hands-on work. 

The next root cause is the normalization of deviances; not questioning anomalies. As 
was evidenced in the Challenger failure, we see deviations in the characteristics that are not 
quite normal, but seem to have no major consequence. After seeing these deviations a few 
times we accept them as normal and ignore them. [Rogers Commission Report, 1986] The 
result is a major failure where the deviation becomes catastrophic. The technical mind must 
question each and every deviation and develop risk-based understanding for operations to 
proceed. The history of space flight is replete with examples such as Challenger (in general 
smaller consequences) of small deviations that have over time become major incidents. 

More will be discussed on this subject later. 

The third root cause is closely related to the second in that it is the lack of critical 
thinking with an over reliance on computers, processes and procedures. Computers, 
processes and procedures are necessary but can never take the place of the human mind. 

In the end all of our resources, tools etc. are there as an aid to the human mind and the 
human creativity and decision making. 

The fourth root cause is an organizational and culture cause where authority was 
decentralized to the extent that there was no real decision making authority and personnel 
did not know where to go to get a decision. This led to a floundering and stifled organization 
that was paralyzed by its inability to get actions. 
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The final root cause is closely related to the fourth and is driven by the technical and 
organizational complexity. Space systems are highly complex and very sensitive to small 
uncertainties. This complexity requires in-depth penetration and understanding where small 
technical glitches result in major problems. The technical complexity leads one to think that 
the technical complexity must be matched with organizational complexity. We know that 
when managing the organizational complexity becomes as large as or larger effort than 
managing the technical, then the product success is in question. Simplicity is the pathway to 
product success both technically and organizationally. 

Listed below are five of the major incidents used in this study for developing the root 
causes and the preparation of the process of achieving excellence in engineering. In 
parentheses are shown the root cause numbers that pertain to this incident. There are many 
more that were looked at; however, these are representative of the problem cause and 
problem solution. The reader can add to the list as they desire. There are many books 
written on failures, failure analysis, etc. “Beyond Engineering, How Society Shapes 
Technology” [Poole R., 1997], is one book that deals with the complexity factor and the 
influence it has on failures both technically and organizationally. It should be required reading 
for managers and for engineers. 

Examples Supporting Root Causes (Numbers in parentheses indicate associated root 
causes) 

1. Space Shuttle Challenger ( 2, 4, 5 ) [Rogers Commission Report, 1986] 

2. Space Shuttle Performance ( 3, 5 ) [Holloway, 1999] 

3. Space Shuttle Columbia ( 2, 3, 4, 5 ) [Gehman, et al, 2003] 

4. Hubble Telescope Mirror (1,2) [Harland, et al, 2005] 

5. X-33 Single Stage to Orbit ( 3, 5 ) [David, 2001] 


Elements of Engineering Excellence 

Taking the incidents studied and the resulting root causes led to the development of 
an approach for achieving excellence in engineering. This approach may be divided into 
three elements* as illustrated in Figure 2. First, Technical Understanding and Execution — it is 
basic that engineering products must be technically correct. Second, Partnership with the 
Project — successful products require a positive, productive relationship between Engineering 
and the Project Office. Third, Individual and Organizational Culture — all activities are 
undergirded by the prevailing culture, which must foster the attitudes and behaviors 
necessary for success in producing and operating our complex systems. 


* The triad is the simplest representation of most solutions. If we can reduce our approach to a triad 
then we have a more easily understood means of solving our problems. 
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Technical Understanding and Execution 

• Understanding the Physics 

• Ensuring Integration 

• Interactions and Interfaces 

• Sensitivity, Uncertainty, Margins 



Partnership With Project 

• Technical Authority 

• Requirements Management 

• Risk Management 

• Cooperative Solutions 


Individual & Organizational Culture 

• Ownership and Accountability 

• Critical Thinking vs. Procedures 

• Right People in Right Places 

• Learning Organization 


Figure 2. Elements of Engineering Excellence 

Each of these elements of the solution has four major divisions that will be discussed 
in detail in the following sections. 
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Technical Understanding and Execution 

While engineering involves aspects of many fields of endeavor, its core is technical 
work. This essential technical work must be done correctly and comprehensively; therefore 
Technical Understanding and Execution is fundamental to Engineering Excellence. 


Technical Understanding and Execution 

• Understanding the Physics 

• Ensuring Integration 

• Interactions and Interfaces 

• Sensitivities, Uncertainties, Margins 


Four aspects of Technical Understanding and Execution will be addressed: 

• Understanding the Physics 

• Ensuring Integration 

• Interactions and Interfaces 

• Sensitivities, Uncertainties, and Margins 


Understanding the Physics 

It is clear that in-depth technical knowledge and expertise is essential and fundamental 
to the engineering process. This is founded on understanding the physics of the system 
(Figure 3). 
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In-depth technical knowledge and expertise is essential and fundamental 


• Technical experts should 



- Step back from the details and take a 
broad view of what is happening 

- Consider the real system vs. the model 
(Don’t eat the menu) 

- Use simplified models to help 
understand the phenomena (If you 
can’t explain it in simple terms, you 
don’t understand it) 

■ Explain the technical concepts and 
significance of the results to the 
leadership 


Figure 3. Understanding the Physics 


Technical experts should focus not only on the details of their technical specialty, but 
should step back from the details and take a broader view. Understand the physics of the 
system-consider what is happening with the real system, and don’t become enamored with 
the model. The “model” can be either the analytical model, the simulation, or the mental 
model that is carried in our mind. Recognize that these are just representations of the actual 
system, and are necessarily incomplete (and sometimes misleading). 

At the same time, models can be necessary tools for our understanding. Use 
simplified models (e.g., back-of-the-envelope approximations, free-body diagrams, etc.) to 
help understand the physical phenomena. Our early mentors strongly emphasized this. 

They correctly thought “If you can’t explain it in simple terms, you don’t understand it”. As a 
corollary to this idea, when communicating technical issues with management, engineers 
owe managers an explanation of the issues and their significance expressed in simplified 
terms. 


How does one acquire an understanding of the physics of the system? In addition to 
using simplified models, take every opportunity to see and touch the real hardware. As initial 
components and systems are built and tested, go see them. Lay hands on them. Visit test 
areas and participate in testing. Visit contactor plants where hardware is being built and KSC 
where hardware is being assembled. If the system you are working on is not yet to the 
hardware stage, search out similar hardware and related facilities. These experiences will be 
invaluable in understanding of what the real system is like and how it performs. 
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Ensuring Integration 

Experience has shown that most technical problems are systems problems that occur 
because of interactions, interfaces, and communications breakdown. It is of primary 
importance that products are fully integrated in their design and operations. Complex space 
systems are composed of many parts, entail numerous diverse technical areas, and their life 
cycle involves a wide variety of processes and events. Also, the system must meet multiple 
requirements and constraints in safety, performance, cost and schedule. All of these 
aspects must be integrated into a balanced system, as illustrated notionally on Figure 4. 



Figure 4. Successful System Requires Integration of All Aspects 


The success of the system depends on ensuring this complex, multi-faceted 
integration. Ensuring that the needed integration takes place is complicated by the common 
natural tendency for individuals to focus on their part or technical area without adequate 
attention to its interactive effects with the rest of the system. While it is necessary to have 
parts and technical areas correctly executed, it is essential that they be fully integrated with 
the rest of the system. This is the basis of the “T-Model” philosophy that illustrates in-depth 
technical execution with an overarching system perspective. 

Successful integration involves multiple approaches, both formal and informal. Formal 
integration includes official instruments such as interface control documents, and dedicated 
organizations such as integration offices and working groups. Informal integration is the 
ongoing communication among technical areas by way of person-to-person interactions, 
discussions, informal meetings, and so forth. Providing means for formal integration is 
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necessary, but informal integration is the more powerful in achieving an integrated product, 
and can’t be overemphasized. Pervasive communications is a vital enabler of integration. 

Those performing the design need to understand where they fit in the 
compartmentalization/reintegration process and the nature of their interfaces and interactions 
as a part the total system. Integration involves finding the correct balance among system 
requirements and constraints, designing for the total life cycle including manufacturing, 
verification, and operations, and bringing together the design and discipline functions for all of 
the hardware and software constituents so as to produce a successful total system. A 
successful system has acceptable performance and is safe, reliable, timely and affordable. 

All must be alert for integration issues and proactive in addressing them, 

Management should organize for integration and create an environment that encourages and 
ensures integration. Integration is everyone’s responsibility. 


Interactions and Interfaces 

Our complex space systems are highly interconnected and interactive. What happens 
in one subsystem or area usually affects other subsystems or areas, often in ways that are 
unexpected. Reintegrating compartmentalized parts frequently produces system behavior 
not anticipated during the compartmentalization. One way of describing this is “Things come 
apart linearly, but go back together nonlinearly.” There are interactive phenomena such as 
flutter and pogo that only exist in the system context. 

Figure 5 illustrates the concept, where elements may perform satisfactorily by 
themselves (or possibly in simple combinations), but when combined with the other elements 
of the system produce unacceptable interactive effects. 
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Interactions between combined system elements can produce 
phenomena not exhibited by individual elements. 


Figure 5. Problem Produced by Interactions Among Combined Elements 


It is necessary to understand and manage Information flow among all the 
compartmentalized parts throughout the design process; however, interfaces alone do not 
identify all the interactions that occur when the parts are combined. As noted above, some 
critical interactions can only be addressed from a system perspective. 

How do we deal with these interaction and interface issues? Engineers and managers must: 

1 . Give much deliberate attention to potential interactions, exploring them at each level of 
integration. Specifically evaluate areas of known interactions through analysis and 
testing. Design-in means of reducing or decoupling the interaction, such as pogo 
accumulators. 

2. Look for opportunities to apply combined analyses, i.e., analyses that formulate the 
combined problem from first principles instead of handing off between separate 
analyses. An example might be combined thermal-structural analysis of ablative 
materials. Similarly, test with as many combined effects and environments as 
possible. 

3. Manage the interfaces and the data flow across them. Know who is responsible for 
each area. 

4. Maintain robust informal communications with all disciplines and technical areas. 
“Robust” here means communicating in multiple ways — don’t have single-point failures 
in the communication chain. As stated previously, formal methods of communication 


10 


are necessary, but informal communications are more powerful in achieving technical 
integration. 

Just as our hardware/software systems are complex, our organizations typically are 
also complex. Organizational interactions and communications can be as complex as the 
technical and require a high level of attention. Organizational structures should not be overly 
complicated, and should streamline information flow as much as possible. It has been said 
that if integrating the organization requires more effort than integrating the technical product, 
clearly something is wrong with the organization. 

Robert Pool [Pool, 1997] has pointed out the need for adaptability in organizations: 
“Layered organizational structure seems to be basic to the effectiveness of 
organizations... some groups are bureaucratic and hierarchical, others professional and 
collegial; others are emergency response. ... Because of complexity, they are best 
decentralized; because of tight coupling, they are best centralized.” Organizations must be 
flexible to adapt to the given situation. “High reliability organizations also emphasize active 
learning, not simply the memorization of procedures.” 


Sensitivities, Uncertainties, and Margins 

An important aspect of technical understanding and execution of design for high power 
density launch vehicles is associated with sensitivities, uncertainties, and margins. These 
quantities enable the designer to achieve the best design concepts because they provide 
important interactive insights. 

Launch vehicle design goes back to the late 1950’s where there was a lack of 
understanding of environments; material and manufacturing characteristics; computational, 
simulation, and testing capabilities; and overall design experience. During the time frame of 
1957 until 2010 there were 344 major launch vehicle failures [Encyclopedia Astronautica, 
2010]. About 28% of the failures had unknown causes (Russian failure causes during the 
50’s and 60’s are not in public domain). The total percentage failures since 1980 for each 
country are: [Russia, 4.2%]; [USA, 6.1%]; [Europe, 5.8%]; [Japan, 9.4%]; [China, 11.6%]; 
[India, 40%]; and [Rest of World; 50%] [Demidovich, 2007], The cost of these failures is in the 
billions of dollars and loss of international prestige. 

From 1990 to 2006 Launch vehicle failures have been attributed to engineering 
mistakes, technology surprises, or a combination of both [U. S. Government Satellite — 
2007], Of the total of 22 failures in that time frame, 7 have been associated with engineering 
mistakes, 10 associated technology surprises, and 5 with a combination of both. The major 
causes of failure by subsystem from 1980 to May 2007 are shown in figure 6 below. 
Historically, propulsion and guidance and navigation systems have been the first and second 
largest causes of failure. 
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Figure 6. Known Causes of Launch Failures Worldwide by Vehicle Subsystem 

(1980-May 2007) 


As the design proceeds, the designer tries to mitigate sources of failure. There are a 
number of ways to accomplish that: catch errors, examine hardware, question modeling and 
testing results, and so on. However, it turns out that determining and understanding 
sensitivities, uncertainties, and margins provide necessary insights into a design along with 
insights into all the interactive aspects of multi-subsystem coupling associated with high 
power density launch vehicle systems. 

Sensitivity factors usually pertain to the change of one variable with respect to a 
change in another variable (partial derivative). High performance systems usually have high 
sensitivities. This is illustrated in Figure 7 where the variation of sensitivity and performance 
can be seen. This illustrated variation is typical and, furthermore, high performance systems 
are usually on the steep part of the curve because high performance demands are necessary 
to meet mission requirements. 



Figure 7. Variation in Sensitivity with Performance 
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An example of sensitivity with performance can been seen in a comparison of the 
power densities (horsepower / pound of engine weight) of automobile engines, jet engines, 
and rocket engines for various power levels. Their power densities are: (car, .54; Indy engine, 
2.91; small jet, 18.3; large jet, 149.3; rocket engine, 879). The higher the performance a 
system has the higher the sensitivity. If an automobile engine could be designed with the 
efficiency of a rocket engine, its weight would be about % pound. The specific impulse 
efficiency achieved by the current design of the SSME is 98%. In comparing these propulsion 
systems, it should be noted that system complexity also increases as performance increases. 

Launch vehicles that put payloads in orbit are high performance systems. This can be 
illustrated through the following three comparisons. Firstly, if a comparison is made between 
horsepower per pound of an automobile, commercial airplane, and a rocket stage, it would 
show that the horsepower per pound of a rocket stage is about 2 orders of magnitude higher 
than a commercial airplane and 3 orders of magnitude higher than an automobile. Secondly, 
if a comparison is made regarding propellant mass fraction (ratio of mass of propellant to total 
mass of system) of an automobile, commercial airplane, and a rocket stage, they would be 
respectively about .04, .4 and .9. This means that a rocket stage is 90% propellant and 10% 
structure + other parts; where an automobile is 4% fuel and 96% structure + other parts. 
Finally, if a comparison is made between the thrust required to fly a man on 1 15 mile 
suborbital flight to an orbital flight to 160 nautical miles, it would take an order of magnitude 
more thrust to go to orbit, e.g. about 800,000 to go to orbit compared to 80,000 pounds of 
thrust for a suborbital flight. These sensitivity comparisons illustrate the differences in the 
various transportation systems and the extreme levels required to be achieved by high 
performance rocket systems. 

Sensitivity factors (partial derivatives) are determined through analysis, test, or 
simulations and can be applied in at least three different ways in design and development. In 
the first, they can be applied to achieve a best balanced design when trying to tune among a 
number of design variables. In the second, when there is hardware in test, sensitivity analysis 
can be applied to confidently make block changes to fine tune operations. In the third, in 
situations where there are high uncertainties in input variables, reducing associated 
sensitivity factors will reduce the uncertainties in the output variables. 

Uncertainties can be determined through historical data bases, test, and expert 
opinion. Prudent designers identify uncertainties and account for them in nearly all aspects 
of design and development. For example, in rocket flight, aerodynamic bending loads are 
proportional to the dynamic pressure (q) multiplied by angle of attack (a). For a circularly 
cylindrical rocket, if there were no wind and no maneuvering of the vehicle, then the angle of 
attack would be zero and thus no bending loads due to aerodynamics. However, there are 
nearly always random atmospheric winds and values of (qa) can be typically between 0 to 
6000 psf-degrees. Because of this uncertainty in the wind, the designer designs for a 3 
sigma wind with a 50 percent confidence level. Then the flight 3 sigma performance 
characteristics (qa) are determined through Monte Carlo simulations [Hanson, J.M. et. al., 
2010]. Quantifying and understanding uncertainty enables risk assessment which provides 
confidence in the designed product. 
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Margins are needed in the design process to provide for unknown uncertainties, 
unforeseen design anomalies, immaturity in the knowledge base, etc. Margins represent the 
difference between an allocated limit and the actual design values including the effects of 
uncertainty. Typically designers are concerned with margins associated with weight, 
electrical power, thrust, specific impulse, etc. They are determined based on historical data, 
maturity of technology, and project phase. Margin tracking provides the project information 
regarding maturity development where it is expected that the margins will reduce as the 
project evolves. 

If the designer doesn’t have a clear understanding of sensitivities, uncertainties, and 
margins, then the design cannot be considered to be robust and results in high risk 
operations. A failed launch could cost in the neighborhood of about $500 million or more. 
However, clear understanding of these quantities provides confidence in the design, enables 
the project to determine the risk with a high level of confidence, and a successful launch 
enhances our international prestige. 
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Partnership with the Project 

The second element of the Engineering Excellence triad is Partnership with the 
Project. Engineering and the Project Office have distinct but interlocking roles that must 
relate in a positive, productive partnership to achieve successful products. This section will 
address productive partnering of the two organizations and will discuss primary areas of 
cooperation between the organizations. 

Partnership With Project 

♦Technical Authority 

♦ Requirements Management 

♦ Risk Management 

♦ Cooperative Solutions 


Four topics will be addressed: 

• Technical Authority 

• Requirements Management 

• Risk Management 

• Cooperative Solutions 


Technical Authority 

There are various ways Engineering and the Project Office can relate, and past 
experience has seen the pendulum swing between the extremes of having Engineering fully 
involved to essentially having “hands off’. Arguably the most productive relationship results 
when Engineering acts as the Technical Authority for the project, serving as the technical 
conscience that ensures the technical correctness of the product. 

What is implied by Technical Authority? It means taking ownership and accountability 
of the technical performance of the product. Ownership produces accountability. There is a 
progression of ownership as illustrated in Figure 8, which takes one from Observer to 
Participant to Owner. The Observer is a spectator who may critique, but still remains 
minimally engaged. Next, a Participant has a “doing” role with shared responsibility. Finally, 
the Owner not only is a doer, but has the full responsibility of ownership. As one moves 
along the progression, there are major increases not only in effectiveness and accountability, 
but also in enthusiasm for the job at hand. 
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Figure 8. Progression of Ownership 

“Automatic Responsibility” was a term used in past years to describe Engineering’s 
ownership and accountability role for the technical aspects in all Center projects, either in- 
house or contracted. It is most productive when this ownership role is internalized (“felt”) and 
asserted by Engineering, and is understood and relied on by the Project Offices, forming the 
basis for technically sound products 

While this approach doesn’t eliminate all conflict between the two organizations, what 
remains is a creative tension that drives toward the best balanced design. 


Requirements Management 

A primary area of partnership between Engineering and the Project is requirements 
development and management. See Figure 9. 
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Figure 9. Requirements Development Responsibilities 


The Project organization holds and formally imposes the project requirements, to 
which Engineering designs the product. However, this is not a one-directional flow. Consider 
the sources of requirements. The customer states top-level mission requirements such as 
what mass is to be delivered to target orbits, delivery schedules, cost limits, etc. The majority 
of the remaining requirements are initially generated by Engineering, including architectures 
and concepts, derived requirements on systems and subsystems, and technical standards. 
These requirements are jointly iterated as necessary with the Project, which then formally 
imposes them. So requirements development is a partnership activity. Likewise, subsequent 
requirements management including flow-down allocation, change management, and 
verification planning entails partnering between Engineering and the Project. 

Unrealistic and technically uninformed requirements have been sources of major 
problems. Engineers should demand to understand the requirements and push back on any 
that are unreasonable. On the other hand, engineers should resist the tendency to “over- 
engineer” the system beyond its practical requirement. Use judgment on when enough is 
enough — if three decimal places are sufficient, don’t insist on six. Over-engineering not only 
is wasteful of resources, but can obscure insight and understanding. Better can be the 
enemy of good. 
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Risk Management 

Engineering in conjunction with Safety and Mission Assurance work together as 
partners with the project to define, accept, and manage risk. Determining and understanding 
technical, cost, and schedule risk throughout the project cycle provides a means to choose 
the best design, focus resources, and provide confidence in the designed product. The main 
focuses of technical risk are safety( personnel, assets, and environmental) and 
performance( requirements, operations, and supportability) . 

Technical, cost, and schedule risk are inherently coupled. As shown in Figure 10 
programmatic risk is constrained by performance requirements, budget, and schedule. 
Furthermore, a change in any of the risks can impact the other two and must therefore be 
assessed. 



Figure 10. Coupled Risk 


Engineering manages, accepts, and owns risk jointly with the project, unless risk is 
technically unacceptable to engineering. An example of being unacceptable to engineering 
occurred on the second flight after the Columbia incident. On the first flight after Columbia, 
debris was liberated from the External Tank. The proposed fixes on the 34 ice ramps were 
not implemented after that flight. Prior to the second flight, the safety panels from MSFC, 
JSC, and KSC rated the risk the highest and recommended not to fly until the fixes were 
implemented. In addition, NASA’s safety and chief engineers recommended grounding the 
fleet until the fixes were implemented. The NASA administrator overruled those decisions 
reasoning any danger would come during reentry. If a problem occurred during flight, the 
astronauts could make repairs before returning, or wait at ISS for rescue by another Shuttle 
or a Russian Soyuz. The flight flew as scheduled and without incident, see reference, 
[Griffin, 2006]. 
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Figure 1 1 provides a typical risk taxonomy after the Design Certification Review 


(DCR). 
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Figure 11. Risk T axonomy after DCR 


By the end of design the risk associated with the risk matrix method should be 
minimized to an acceptable level as indicated in Figure 11. Furthermore the risk associated 
with PRA should also be reduced along with the uncertainty. 

As can be observed, risk assessment and management are necessary in projects 
where there are high power densities and complexities. It enables the best design decisions: 
e.g., picking configurations, and focusing resources. It also provides confidence in the design 
and operations. Determining risk throughout the design process is mandatory; however 
complex it may be. As the design progresses, knowledge of the system and various 
subsystems matures and the risk value and associated uncertainty decreases. Risk 
assessment is determined in a consistent and conservative fashion. 

Cooperative Solutions 

Engineering interacts with the project by cooperating with them in providing 
engineering solutions in two ways. In the first, engineering provides design solutions that are 
balanced and meet the requirements and constraints. These solutions are the result of trade 
studies where technical, cost, and schedule are proactively considered. An overview of the 
trade studies is presented to the project and the rationale for selecting the final design is 
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delineated. Then the selected design is explained in detail along with the associated risk 
assessment. In the second, engineering provides solutions explaining complicated physical 
phenomena. Again, engineering must be proactive in regard to explaining complex 
phenomena in a simplified fashion to demonstrate fundamental understanding and to 
communicate their knowledge to a broader cross-section of the engineering community. 
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Individual and Organizational Culture 

The third element of the Engineering Excellence triad is Individual and Organizational 
Culture. Culture is the set of behaviors and beliefs that form the basis of our individual and 
collective actions. It can be a source of our failures, or can be the foundation that undergirds 
our successes. Product success requires creating and nurturing the individual and 
organizational culture necessary for Engineering Excellence. 


Individual & Organizational Culture 

• Ownership and Accountability 

♦ Critical Thinking vs. Procedures 

♦ Right People in Right Places 

• Learning Organization 


Four of the many elements of successful culture will be addressed: 

• Ownership and Accountability 

• Critical Thinking vs. Procedures 

• Right People in Right Places 

• Learning Organization 

Ownership and Accountability 

The merits of Engineering taking ownership and accountability for the technical 
aspects of the projects were discussed previously. Here we will address the culture of 
ownership and accountability. In building this culture, there are roles for the leadership, for 
individuals, and for teams. 

First, consider the role of Leadership . The main goals of leadership are (1 ) producing 
a successful product and (2) developing people — teaching, mentoring, etc. In pursuing the 
first goal-producing a successful product-it is important that line managers know critical 
issues and ensure that they are being appropriately worked in the organization. Technical 
managers are more than administrators — they have responsibility for the success of the 
product. Secondly, in developing people, leaders should foster and encourage ownership 
and accountability in individuals and in teams. Positive expectation and reinforcement are 
powerful principles in guiding their development. Leaders’ attention to these two essential 
functions should not be displaced by administrative “urgencies”. 

Next, consider the role of Individuals . Five characteristics of individuals who take 
ownership are: 
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• Competence - Develop and enhance my skills and capabilities 

• Cognizance - Penetrate understanding of the design with critical thinking 

• Commitment - Be dedicated to my task 

• Accountability - Be responsible for my product 

• Passion - Feel the drive of enthusiasm and ownership 

Finally, consider the role of Teams . With proper leadership, a team can function at a 
level higher than the level of its most capable member. High performance and ownership is 
attained in teams where the members hold one another accountable, encourage poor 
performers to improve, and establish respect among team members who are held to the 
same high standards. A well-functioning team quickly identifies potential problems by 
questioning one another’s approaches without hesitation. Team members should be free to 
state their opinions, and to explore and challenge each other’s opinions without feeling 
threatened by the dialogue. This characteristic requires an environment of openness. 
Openness and a non-threatening environment result when team members trust each other. 
Trust is built on a foundation of integrity of the team members. Integrity means, among other 
things, consistency between actions and words. Trust will be developed over time when 
team members have Integrity, and Trust will lead to the environment of Openness necessary 
for a well-functioning team. So Openness is based on Trust which is founded in Integrity. 
Open, well-functioning teams promote the ownership and accountability culture. 

Well-functioning teams converge to appropriate answers. After all opinions have been 
heard, it is time for a decision. Someone must be in charge. The team leader should provide 
clear focus and make the necessary decisions. Building ownership and accountability 
requires cooperative efforts among Leadership, Individuals, and Teams. 

Critical Thinking vs Procedures 

Critical thinking is fundamental for the success of individuals and organizations. It has 
been studied and debated for over 2500 years. In fact, it was inspired by the early Greeks 
(Socrates) and evolved through the Middle Ages and Renaissance periods until today. As a 
measure of interest, an Amazon search dealing with critical thinking revealed over 38,000 
citations of books written on the subject. Interest in critical thinking is in almost every 
profession and each has its own tailored definition. However, if these definitions are studied 
there are significant similarities. A straightforward top level definition paraphrased from 
reference [Clayton, 2009] is: 

Critical thinking is the ability to understand, to make and carry out informed decisions, 
and to create, invent, and discover by efficiently utilizing a lifetime of knowledge, 
experience, common sense, reasoning, intuition, feelings, confidence, and so on. 

The purpose here is to illustrate how critical thinking can be typically applied in the 
design of complex engineering projects. In these projects every participant must believe that 
he/she is a critical thinker and that their inputs and questions are valuable. 
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In the design of complex projects, there are fundamental elements associated with 
critical thinking. Some insights associated with the elements “characteristics of individuals 
and organizations” are described below. 

“Insights regarding the individual” 

1 . Confidence in design requires knowledge and understanding of how things work 
including the physics, kinematics, interfaces, interactions, ... 

2. Efficient and effective design means not solely relying on historical procedures, 
processes, codes ... but also thinking through the present application, understanding 
underlying principles and assumptions, and if they do not fit the present needs, tailor 
them for current application. 

3. Risk can be reduced by challenging all assumptions associated with analysis, test, and 
simulations; assessing sensitivities and uncertainties of results; and quantifying 
margins. 

4. Innovative products can be developed by encouraging “critical thinkers” to ask 
questions and think out of the historical box. 

5. A characteristic of highly motivated individuals is their passion for achieving the best 
balanced design no matter the difficulty; they can’t be discouraged; and their passion 
is contagious. 

6. Critical Thinkers are always improving their skills. This includes their 
professional/technical, cultural, personal, and spiritual skills. 

In summary, the above list is meant to provide “insights regarding individuals” and not be 
all inclusive. Furthermore, it can be seen that the above are integrated, synergistic, and 
focused on developing the best balanced products. 


Considerations will now focus on insights related to the organization. 

“Insights regarding the organization” 

1 . The vision for all elements of an organization must be clear and focused. 

For example, during the development of the SSME there were 38 incidents that 
cost more than $30 million each. In many of these, flow induced effects were 
driving factors. As a consequence, it was decided to implement computational 
fluid dynamics (CFD) into the design process. The direction from management 
was to “make CFD a design tool.” 

2. Technical integrity underlies spirit and motivation. Leaders set example; they 
teach/mentor regarding principles, expectations, discipline, and so on. In addition, they 
inspire the search for truth and best balanced design. 

3. Best balanced designs are achieved by questioning all aspects related to technical, 
cost, and schedule issues. Designers should expect questioning and be prepared to 
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present design options and associated risks along with their sensitivities and 
uncertainties. 

4. Highly interactive feedback should be promoted at all levels of design to stimulate 
technical integration. 

5. Critical Thinkers should be recognized for their contributions to the project. 

6. An environment should be established that supports and encourages creativity and 
innovation; eliminate the fear of failure. 

7. Project communication, e.g., technical integration, can be achieved via peer 
interactions and formal/informal reviews. 

In summary, these in addition to others, are important “insights regarding the 
organization.” They indicate the scope and categories of organizational elements to inspire 
critical thinking. 

In addition to fundamental elements related to individuals and organizations, another 
fundamental element of critical thinking pertains to “asking questions”. The focus will be on 
individual and organizational questions to induce critical thinking. 

In considering questions related to individuals (or design teams), the major focus will deal 
with technical questions leading to a best balanced design in which the level of risk is as low 
as reasonably possible. 

“Individual’s technical questions to induce critical thinking” 

(Typical list) 

1 . When are results needed (what milestones)? 

2. What are the requirements? 

3. Does the architecture make sense or should it be changed? 

(What can go wrong?) 

4. What is the design strategy? 

5. What design criteria was applied? 

6. How were failure modes determined and what are they? 

7. Do attributes meet requirements? 

8. What is the maturity of input data? 

9. What is the source of input data? 

10. What are the underlying physics? 

1 1 . What methods were used and how were they benchmarked? 

12. What assumptions were made and what are their associated sensitivities? 

13. What has been the experience using the methods? 

14. How do the results compare to the historical similar cases? 

15. What are the sensitivities, uncertainties, and margins? 

(Understand the pedigree of results) 

16. How are results being used and what are their impacts? 

17. What do the hardware and test data tell us? 

18. What are the underlying patterns in the data? 

19. How were interactions assessed? 
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20. Have interfaces been adequately evaluated? 

21 . How were all the results used in the risk assessment? 

22. Are risk mitigation approaches adequate? 

23. Is there a verification plan and how is verification achieved? 

24. Have the results been assessed by peers? 

25. Is your intuition satisfied; are you comfortable with the results? 

These questions and others enable designers to understand and make the best 
informed decisions to achieve a balanced design where the risk is as low as reasonably 
possible. Asking questions is an important element of critical thinking. This should be 
understood by all critical thinkers, i.e. all designers, and encouraged at all levels of the 
project. 

Another category of questions pertains to organizational questions to induce critical 
thinking. In this category, the focus is on what is being done, how it’s being done, what are 
the options, etc. The questions below are from NYPD Compstat activities. 


“Organizational questions to induce critical thinking” 

1 . What are/were we trying to accomplish? (The Objective) 

2. How well did we/you execute the plan? (Administrative) 

3. What happened? (Descriptive) 

4. Why do we/you think it happened? (Diagnostic) 

5. What alternatives are there? (Creative) 

6. What do we/you think will happen? (Predictive) 

7. What is the best choice? (Evaluative) 

An assessment of these questions indicates that question one pertains mainly to 
“planning the work” and questions two through seven pertain to “working the plan.” 

In overall summary, critical thinking is inherent in all aspects of design of complex 
engineering products. The purpose here is to illustrate how some elements of critical thinking 
provide confidence and fidelity in the final product. The intention is not to provide an all- 
inclusive review of critical thinking; but provide enough insights to encourage designers to 
review the literature and include, as needed, other elements of critical thinking into their 
design domain. 


Right People in the Right Places 

A principle that is emerging deals with the role of a person’s talents and/or “voice” -- 
some would use the word “calling”- in organizational success or efficiency. There are 
several books, three of which are referenced, which deal with the subject. Stephen Covey’s 
“The Eighth Habit”, [Covey, 2004] “Good to Great” by Jim Collins, [Collins, 2001] and “Now, 


25 



Discover Your Strengths” by Marcus Buckingham and Donald O. Clifton [Buckingham et. al., 
2001]. Collins says, “He first got the right people on the bus (and the wrong people off the 
bus) and then figured out where to drive it. Look, I don’t really know where we should take 
this bus. But I know this much: If we get the right people on the bus, the right people in the 
right seats, and the wrong people off the bus, then we’ll figure out how to take it someplace 
great.” Getting the right people on the right bus in the right seat is responsibility of both the 
employees and management. The probing question is: How do I get the right person in the 
right seat on the bus? Remember: 

• Each person’s talents are enduring and unique. 

• Each person’s greatest room for growth is in the area of his or her greatest 

strength. 

• Find your voice, inspire others to find their voice. 

• Discover your strengths. 

• The strength of an organization is in its people, all else are aids to the human 

mind. 

• Selection is a joint employee and organizational task. 


It is not only important to get the right person on the right seat on the right bus, but each 
individual has a responsibility to find their own voice. The following list was a Boeing ad that 
appeared a few years ago that contained some fundamental questions that a person could 
ask themselves that would help understand what drives or motivates a person. 

Motivation 

There is a person inside me who wants to: 

■ Design a church 

■ Run for office 

■ Learn to cook 

■ Build the Space Station 

■ Write a song 

■ Counsel youth 

■ Climb a mountain 

■ Live in Spain 

-Boeing Ad in Launchspace magazine 

What motivates a person is one key to what their strengths are. Other questions are: 


What are my interests? 

- Family? 

- Professionally? 

- Socially? 

- Avocationally? 


26 



What do others ask me to do? Expect of me? Reinforce in me? 

- Family? 

- Professionally? 

- Socially? 

- Avocationally? 

What do I see that needs to be done? Your calling is what you recognize as needs. 
What pulls me (calling)? 

What do I admire in others? 

Who are my role models? 

When Robert Ryan was a student a Peabody College the Head of the Psychology 
department of Vanderbilt University presented the following figure (Figure 12) that illustrates 
the two main paths that an individual can take in his or her development. The choice is 
having versus being. Having is a social emphasis while Being is spiritual in nature. In 
general from a personal and organizational standpoint the selection of being is the obvious 
choice. It has been said by many that if one chooses being and becomes something of 
value, adding meaning to themselves and society, then the having will take care of itself to 
the extent all basic needs are more than met. Having focuses on position, money, authority 
and accolades while Being focuses on values, spiritual dimensions and adding meaning to 
society as well as to oneself. 


Focus 


Success 


Substance 


Social 


Personal 


Having Being 

Figure 12. Two Paths of Development 


Another imperative that the same professor gave is illustrated in Figure 13, called the 
wheel of life. Each individual’s life is composed of spiritual, mental, physical, social and 
emotional components which should be balanced to the degree possible. What happens is 
that an individual concentrates on their strengths and ignores their weakness. The better one 
can balance these areas the smoother life can be, yet our greatest room for growth is in our 
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areas of strengths. The paradox is obvious in that we grow by working on our strengths, the 
area of greatest value to an organization, yet we must somehow work on weakness in order 
to have a better balanced life. It is called the principle of compensation in that we build on our 
strengths to overshadow our weakness. In doing this our life is increasingly more unbalanced 
and does not run as smooth. In general one cannot get a total balanced set; however, if one 
is aware of the principle and what is taking place, coping is much easier. 


The Wheel of Life 




Must understand the principle of compensation (overcompensation for weakness), 
balancing out the five areas of life to the degree possible, then accepting the results. 


Figure 13. Wheel of Life 


The principle of getting the right people in the right seats on the right bus [Collins, 
2001] dictates that each individual must be where their strengths, talents and, as Covey says, 
their voice directs what they be [Covey, 2004], A few questions and ways of finding that voice 
have been discussed in the previous paragraphs and are guides to help implement the 
principle. In summary, quoting from Covey’s book, is the final pearl of wisdom: 

“Find your own voice; help others find their own voice. ” - Stephen Covey 

The next overriding principle in the process of getting the right people in the right seat 
on the right bus is the principle of Choice, both individually and organizationally. Figure 14 
shows the principle as coming to the forks of the road. In New Hampshire, where Robert 
Frost once live in a house at the foot of the hills there is a path which leads from the house 
through bushes. The path comes to a fork where one fork in the path leads into the foothills 
and the other leads into a meadow. There is a tree sitting in the point of the fork and on that 
tree is Frost’s poem “The Road Not Taken”. The story is told that as he was heading down 
the path one day and came to the fork of the paths he got the inspiration for the poem. If you 
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are ever in that region go by the house and walk the path. The message is clear: at any one 
instant you can only take one of the paths of the fork in the road, which is choice. 


Choice 

Where You Go Depends on the Choice You Make 



The Forks of the Road Choice 
Figure 14. The Forks of the Road Choice 


There are six motivations which drive how a person makes a choice, as illustrated on 
Figure 15 [Covey 2004], shown in ascending order to the highest level of motivation. The 
lowest way to make a choice is motivated by anger and results in rebellion or quitting. The 
next level of motivation is to make a choice out of fear and is called malicious obedience. 
Next up the list is doing what I do for the reward it gives and is called willing compliance. 

Duty is the next higher motivation and is fundamental in many of things we do such as 
service to country, community etc. It is normally referred to as cheerful cooperation. Love is 
a very high motivational approach to making our choices, associated with heartfelt 
commitment. The highest level of motivation is adding meaning to the organization, the 
individual and society and is associated with creative excitement. It is very important that as 
an organization and as individuals, we work diligently to cluster our motivations for choices in 
the top three and make basically none on anger and very few based on fear. Using meaning, 
love and duty as our primary motivations builds the individual, the organization and the 
culture. 
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FREEDOM AND POWER IN PEOPLE TO CHOOSE 
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Figure 15. Motivations for Choosing 


The organization has a major role in helping individuals find their voice, developing 
their voice and placing them in the organization based on their voice. Of prime importance is 
for the organization to keep a constant focus on recognizing each employee’s unique gifts 
and capabilities. Once management has recognized the individual’s unique gifts and 
capabilities their next task is to put them in the right place in the organization to utilize those 
characteristics. The final responsibility of management is to foster their continued growth. 
Deming in courses several years ago said, “Management should approve any course an 
employee wants to take even if it does not have direct application to their job.” Developing the 
whole person has organizational impact due to the individual’s development. The quote by 
Albrecht should be an underlying principle for any management and organization. 

“ When an employee shows up for work, you’ve already purchased his or her. . .IQ points, 
or at least you have an option on them. At the end of each day, you have either exercised the 
option or you’ve let it expire. That day will never come again, and the option on that day’s IQ 
points is gone forever. ” 

- Karl Albrecht, quoted in Faster Learning Organization [Guns, 1996] 


Learning Organization 

The method by which we implement the principles just discussed for getting the right 
person in the right seat on the right bus and for developing both the organization and the 
individuals in the organization is through “A Learning Organization.” The following banner 
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(Figure 16) states that the secret of moving into the future is for the organization to become a 
Learning Organization. The question faced is: What is a Learning Organization? 


“A secret for moving into the future is for the 

organization to become a Learning Organization. 



Figure 16. Learning Organization 


There are many viewpoints of what constitutes a learning organization. We have 
studied many of the characterizations and selected the Senge model of a learning 
organization to discuss as an example [Senge, 1990, 1994], The following is a discussion of a 
Senge learning organization model. 

Peter Senge and others have looked at the complexity of current organizations and the 
speed that technologies are evolving in our culture. He and others find that the world is a 
world of fragmentation or compartmentalization. We grow up in a world that continually 
teaches us to fragment our world and establish boundaries. This results in teaching us to play 
roles to fit the situations. Compartmentalization eventually leads to losing connection to the 
larger whole and a feeling of loss. Trying to assemble these fragments is like trying to 
assemble the parts of a broken mirror, in which case it is better to get a new mirror. It is 
concluded that the only means of survival is that the organization and the individuals become 
learning organisms. The following figures are taken from a course on “Learning 
Organizations” developed by Robert Ryan and Coy Brown that show the essence of Senge’s 
“Learning Organization”. The artwork of the figures was done by Coy Brown. 

Quoting Senge: “Today, I believe, five new ‘component technologies’ are gradually 
converging to innovate learning organizations. Though developed separately, each will, I 
believe, prove critical to the others’ success, just as occurs with any ensemble. Each provides 
a vital dimension in building organizations that can truly ‘learn,’ that can continually enhance 
their capacity to realize their highest aspirations.” He asserts that these emerging and 
converging disciplines work together as a system which if implemented will grow and keep an 
organization viable. Figure 17 illustrates the integration of the five disciplines, with systems 
thinking being the fifth discipline and the integrator. 
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The Five Learning Disciplines 
are Gradually Converging 



Figure 17. The Five Learning Disciplines Converging to the Whole: 

The Learning Organization 


The discussion that follows will be only an introduction to learning organizations. The 
reader is encouraged to explore books from the reference list to dig deeper into the subject. 
We will now discuss the characteristics of each of its five disciplines: mental models, shared 
vision, personal mastery, team learning, and systems thinking. 


A. Mental Models 

“What you see is what you expect to see” is a truism repeated over and over. Senge 
says, “Our mental models determine what we select and thus the personal and organizational 
growth.” We have all played the party game where a rumor is whispered in someone’s ear. 
That person then whispers what they have heard, to the next ear. The process is continued 
until the end of the line is reached. The last person now tells what has been heard. In 
general it nowhere comes close to the original statement. Eyewitnesses to an accident will 
see different things. The problem is that we are not aware of these tendencies to modify our 
perceptions by our mental models. Mental Models are tacit, existing below the level of 
awareness, and therefore they are often untested and unexamined (Figure 18). For 
individuals’ and organizations’ growth these models must be brought to the surface for 
exploration and discussion to see how they (the lens through which we view things) impact 
our lives. 
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Mental Models 


Mental models are the “pictures” we carry in our 
heads... the way we look at life and work... our 
personal framework for how we make sense out of 
the world, and how we take action. 

Our mental models are deeply ingrained assumptions, 
attitudes, generalizations, pictures, or images that influence 
our behavior. 


Very often we are not consciously aware of our personal mental models, or the 
effects they have on our behavior; but, in fact, we are seeing the world through 

our own personal lens. 



The skills of Reflection, Inquiry, and Dialogue, which help us surface and 
evaluate our mental models, are essential to effectively analyzing and taking 

action on critical issues. 

Figure 18. Characteristics of Mental Models 

A concerted effort must be made to find ways to reshape the lens and find ways to 
better focus the true image (truth). To grow then we must change who we are, our 
paradigms, in order to see reality, the truth. Knowledge and information are critical 
commodities of the future. Future success is much more likely when continuous learning 
becomes an integral part of the culture of an organization. [Senge, 1990, 1994] Mental 
models are one of the keys to the learning process. 

The pitfalls we have in surfacing and evaluating our mental models are; 

1 . Our beliefs are the T ruth 

2. The Truth is obvious 

3. Our beliefs are based on real data 

4. The data we select are the real data 

He goes on to say that the approach taken must balance advocacy with inquiry— that 
we must lay out our reasoning and thinking and encourage others to challenge it. This means 
that the only way to change is make yourself vulnerable. Robert Hargrove lays out problems 
incurred in balancing advocacy and inquiry. [Hargrove, 1995]. This is a very difficult task 
since we all have the tendency to become defensive; therefore we are not open to the 
challenge. 

We must develop the skills of how to balance advocacy and inquiry for self 
development, thus organizational development. The only way an individual and thus an 
organization can grow is by changing the individual’s mental models. Otherwise everything is 
heard and interpreted in terms of our mental models. Senge uses the Ladder of Inference as 
the model of how we quickly jump to an interpretation of the situation based on the mental 
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models. We can change mental models through reading, visuals etc. but is done most 
efficiently through dialogue. 


Figure 19 depicts four different top level categories of mental models. The lowest is 
low advocacy and low inquiry where the person is withdrawn and silent who does not have 
any ideas, views or even any questions. This is saddest of all people. For example, when 
teaching high school we had a young lady that never had a question, a thought or any kind of 
statement. She was unresponsive in any adequacy inquiry situation and showed very little 
signs of growth. All the teachers tried to reach her and bring her out but with very little 
success during the time we were teaching there. This is probably the saddest situation we 
have encountered where the personality is not open to advocacy and inquiry. It is very hard 
to lead something that is not moving in some direction. In contrast the person who is always 
only advocating tends to dominate and suppress the others in a group. People that just ask 
questions without any views can also dominate and stagnate a group in achieving learning. 
The best is a balance between advocacy and inquiry, where each states their views and is 
then open to inquiry from others. This dialogue results in a change in mental models and is a 
very powerful growth engine. This balance is what every organization and group should 
strive for. 


Advocacy and Inquiry 


LOW ADVOCACY / HIGH INQUIRY 

* Asking questions but not 
revealing your views 


HIGH ADVOCACY / HIGH INQUIRY 

* Communicate: Conclusions, data, 
steps in your reasoning 


* Inquire to discover steps and 
gaps in one another's reasoning 


LOW ADVOCACY / LOW INQUIRY 
* Not revealing your views or 


HIGH ADVOCACY / LOW INQUIRY 

*Advocating your view but not 
inquiring into others’ views 


questioning other's views 
* Silent withdrawal 



Figure 19. Categories of Advocacy and Inquiry 
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Outdated mental models can hinder learning. Many insights into new markets or 
improving outmoded organizational practices fail to get into practice because they conflict 
with powerful, tacit mental models. There are three questions an organization must ask: 

1 . What are some of the powerful mental models that exist in our organizations 
currently? 

2. How are existing mental models affecting the way we practice our business? 

3. Should we change any of our mental models? 

Remember: 

Outdated Mental Models Can Hinder Learning 


The following Talmudic teaching sums up the discussion on mental models. 



B. Shared Vision 

Experts all agree that vision, personally and organizationally, are the foundation of 
growth and excellence. The early prophets stated in the Bible, “Without a vision the people 
perish.” Others have stated the principle in many ways all alluding to the primacy of vision. 
Senge treats shared vision extensively. The following is a short summary of this treatment. 

Enabling organizations to truly become a learning organization and succeed in today’s 
world of “Whitewater” there must be a shared vision (Figure 20). The shared vision emerges 
from personal vision. The shared vision implies that both the organization and the individual 
must make adjustments of their visions in order to have a true-shared vision. There are 
various analogies of organizational shared vision, the hologram being the most powerful. As 
in a hologram, if you start with the whole and cut it up in pieces the image does not 
fundamentally change. In reversing the process by adding up (combining) the pieces the 
whole does not fundamentally change. Organizationally the implications of this model are 
clear. 
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Shared Vision 




CoNVeXSATto/J 




• Shared Vision 

-Alignment of values 
g) - Empowered people 
- Inspired workforce 
- Involved and committed leaders 




Figure 20. Shared Vision 


The following is a list of characteristics of shared vision. 

• Shared visions emerge from personal visions. 

• The hologram is the example. When you add up the pieces, the whole (image) does 
not change fundamentally. 

• Requires that you “hold” the vision while remaining committed to seeing current reality 
clearly. 

• Involves 

- Telling 

- Selling 

- Testing 

- Consulting 

- Co-creating 

• Must be based on the purpose of the organization. Focused around building shared 
meaning. Collective sense of what is important. 

• A shared vision implies commitment. 

The shared vision is collective sense of what’s important based on the purpose of the 
organization. It involves telling, selling, testing, consulting, and co-creating. Obviously a 
shared vision implies commitment. Commitment has four pillars of support as illustrated on 
the Figure 21 [Kinlaw, 1993]. Clarity is required for commitment. In general one cannot truly 
commit to something not understood. Competence is another key. Each individual must 
bring their skills/competencies to bear in a support to the shared vision. This is a key part of 
commitment as is influence. Appreciation of both the individual and team efforts supplies the 
energy for continued growth and performance. 
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The Supports of Commitment 



Dennis Kin law 


Figure 21. Supports of Commitment 


Clearly shared vision is important to a learning organization. The shared vision 
provides the dynamic tension between the current reality and what we are going to be. The 
shared vision keeps the organization and the individuals focused on what they are becoming. 


C. Personal Mastery 

Personal mastery is the thing we appreciate so much in watching performances. For 
example, Michael Jordan, with his exhibition of the skills and mastery of basketball, is 
something to behold. The grace, flow, and ease at which he appears to perform are a 
testimony to his dedication and practice in obtaining the mastery he displays. But the big 
element of personal mastery that led to the development of all these skills was his ability to 
master himself and dedicate his energies to mastering these skills. The same can be said of 
any individual who masters his/her field, or the performance of a group such as the Boston 
Pops or the Philadelphia Symphony. The orchestra is a good model to illustrate both personal 
mastery and shared vision. The shared vision is the production of a great composition, 
composed by someone renowned. The leader or someone else may rearrange the original to 
produce a different production (sound) and display some of the individual skills of the 
members. The commitment to the arrangement is the shared vision. Personal mastery is the 
key: however; each individual member must play their part to near perfection or the clear 
harmonious sound is not present. In addition, regardless of the personal mastery achieved, 
the individual cannot dominate the presentation or again the blend is destroyed. The beauty 
is the blending between the strings, the winds, the percussion, etc. A second factor comes 
into play when someone achieves such personal mastery. They raise the level of 
performance of everyone else on the team, the orchestra, and the organization. 
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The story is told that when Knute Rockne had one of his great teams at Notre Dame, 
the backfield (The Four Horsemen) was outstanding. They became arrogant and cocky and 
looked for all the praise, downplaying the efforts of the other team members. In a critical 
game where Notre Dame was moving the ball down the field with ease, Rockne removed the 
first string lineman, substituting the second string. All of a sudden the backs were being 
tackled in the backfield. They could not move the ball down the field. Rockne let the situation 
repeat for awhile before inserting the first string lineman back into the game. The story goes 
that Rockne never had any more trouble with the Four Horsemen again. Personal mastery 
committed to another vision or goal than the one the team is playing always results in poor 
team performance. Personal mastery committed to the shared vision results in high 
performance. 

Commitment to a vision without personal mastery also results in poor performance. 
Personal mastery is fundamental to success of a team and a learning organization. Personal 
mastery is the thing most obvious and inspiring of all the characteristics of learning 
organizations. Senge deals extensively with the characteristics in the references and in 
Figure 22 which summarizes some of his points. 

Personal Mastery 

Personal Mastery is the discipline of 
continually clarifying and deepening our 
personal vision, of focusing our energies, 
of developing patience, and of learning 
how to see current reality more clearly. It 
entails a level of proficiency and involves 
as P' ra ^ on - 



Figure 22. Personal Mastery 

As evidenced by the examples given, organizations learn only through individuals who 
learn. Individual learning/mastery is based on clear principles and is grounded in 
competence and skills but goes beyond. “It means approaching life as a creative work, living 
life from a creative as opposed to a reactive viewpoint.” [Senge, 1 990, 1 994] As Senge and 
others have so clearly stated, the development process is fueled by a dynamic tension 
between the personal vision and reality, which is the source of the creative energy. 

“The essence of personal mastery then is how to generate and sustain creative 
tension in our lives. This is anchored by personal vision, which focuses on ultimate intrinsic 
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desires and not secondary goals.” [Senge, 1990, 1994] Visions must be in alignment with his 
or her purposes-a sense of why he or she is alive. Based on a vision aligned to purpose, 
meaningful goals can be developed. Senge and others emphasize that personal mastery is 
the integration of reason and intuition. The role of emotional intelligence has been dealt with 
in [Weisinger, 1998] and by many others, detailing its importance in growth. Personal 
mastery is always a commitment to the whole discussed previously and teaches choice. 
Choice is fundamental in all of life processes. Each person is unique and has a special task, 
a calling, to fulfill; if not developed and performed it cannot be replaced. 

In the end the prime characteristics of personal mastery are the control and focusing of 
their energy on the vision of their being, growth, and its application to the organizational 
vision and the current task. 


D. Team Learning 

Much has been written on teams and teaming, for example in references [Martin, Don, 
1993] and [Rich, B. 1994], It is the in-vogue thing to do today-everyone has teams. 
Organizationally, learning must be both individual and collective as illustrated previously. 
Senge has said, “Collectively we can be more insightful, more intelligent than we can possibly 
be individually. The IQ of the team can potentially be much higher than the IQ of the 
individuals. ’’[Senge, P. 1990, 1994]. The teaming idea is not new however. In the Old 
Testament, Ecclesiastics 4; 9-12, says, “Two are better than one; because they have a good 
reward for their labor. For if they fall, the one will lift up his fellow: but woe to him that is alone 
when he falleth: for he hath not another to help him up. Again, if two lie together, then they 
have heat: but how can one be warm alone. And if one prevail against him, two shall 
withstand him; and a threefold cord is not quickly broken.” If one is open to the exchange, the 
vulnerability, growth is essentially unlimited. Clearly certain functions can only be performed 
as a team. Basketball, the symphony, the design and building of a high performance aircraft 
are a few examples of teams. Figure 23 outlines the characteristics of team learning. 

Team learning is the process of aligning and developing the capacity of a team to create 
the results its members truly desire. It is a discipline that requires a commitment to 
continuous study of the interpersonal group dynamics and the team problem-solving tools. 
The discipline includes becoming skilled in slowing down our thinking processes and 
observing and analyzing the group “process”— becoming aware of ways to enhance clear 
communication. Figure 24 illustrates the power of alignment, as does the laser beam. 
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Team Learning 


Team Learning is Vital for Organizational Health. When Teams are Truly 
Learning, They are Producing Extraordinary Results and the Individual 
Members are Growing More Rapidly Than Could Have Occurred 
Otherwise. 


“Dialogue” is the Capacity of 
Members of a Team to Suspend 
Assumptions and Enter Into a 
Genuine “Thinking Together” Allowing 
the Group to Discover Insights Not 
Attainable Individually. 



Figure 23. Characteristics of Team Learning 




Individual Vision 


Team Vision 

(Unfocused) 


(Alignment) 


Figure 24. Team Vision Alignment 


Team learning is characterized by dialogue that causes one to become observer of 
their own thinking, self-mastery/self knowledge that involves looking to knowledge of, 
alignment with others on the team, suspends assumptions in order to honor the position of 
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other viewpoints, is a trusting relationship, and emphasizes diversity with unity. [Senge, P. 
1990, 1994]. Team learning requires a dedication, commitment, sacrifice, respect, 
appreciation, and alignment. When these are present, learning and accomplishment are 
unprecedented. The focusing of the human minds as teams on a vision and goal (solving 
crippling problems) heightens the receptivity, creativity, and insight of each individual on the 
team. The power/leverage in organizational and personal life is in the balancing between 
conflicting requirements, or said differently, dealing with the paradoxes of life. 

The following is a set of characteristics of team learning. 

• “Collectively, we can be more insightful, more intelligent than we can possibly be 
individually. The IQ of the team can, potentially, be much greater than the IQ of the 
individuals.”— Senge 

• Requires dialogue. In dialogue, people become observers of their own thinking. 

• It starts with self-mastery/self-knowledge, but involves looking outward to knowledge 
of, and alignment with, others on your team. 

• Suspends assumptions to honor the passion of the other person’s viewpoint. 

• It is a trusting relationship. 

• Emphasizes diversity with unity 

• "Alignment of each to achieve a common goal. 


E. Systems Thinking 

The fifth element of Senge’s Learning Organization is Systems Thinking. [Senge, 
1990, 1994] (Figure 25) The scope and impact of systems thinking is so broad and 
important, that the reader is encouraged to study it in depth. System thinking is a key to 
many elements related to organizations, individuals, engineering, etc. It takes many forms. 

It has been our experience that 80% of the time the root cause of engineering problems and 
failures is not in various disciplines but is a breakdown in systems/technical integration. 



System Thinking: 

Parts of the Whole System Influence 
all the Other Parts, and That 
Influence is Often/Usually Hidden 
From View. (You Gotta Look for it!) 


Figure 25. Systems Thinking 
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Senge provides some of the characteristics of systems: 

• Structure Influences Behavior 

- "The System causes its own behavior" 

- When placed in the same system, people, however 
different, tend to produce similar results. 

• Structure in Human Systems is Subtle 

• Leverage Often Comes from New Ways of Thinking 

■ Systemic Structure (Generative) 

■ Patterns of Behavior (Responsive) 

■ Events (Reactive) 

• Business and Human Endeavors are Systems , Bound by Invisible Fabrics 
of Interrelated Actions. 

• Systems Thinking is a Conceptual Framework, a Body of Knowledge and 
Tools That has Been Developed Over the Past 50 Years, to Make Full 
Patterns Clearer, and to Help Us see how to Change Them Effectively. 

Essentially system thinking pictures a process with every element/parameter a cause 
and effect. Therefore system thinking starts with the concept of feedback. The feedback 
can exist as reinforcing loops (engine of growth and instability) and balancing loops 
(stabilizing loops). [Senge, 1990, 1994]. 

It is very important to understand, appreciate and balance between the reinforcing loop 
and the balancing loop. Senge gives many examples of these two loops. Key to this 
understanding is the inherent characteristics of delays or lags. 

The nemesis to system thinking is compartmentalization, fractionalization, and 
specialization. Although it is necessary to have compartmentalization, fractionalization, and 
specialization., they must always have a system focus or viewpoint or things like the tragedy 
of the commons will occur. [Senge, 1990, 1994] 

System thinking then is the coordinator and integrator of the system that makes it play 
together as a whole, yet is a fundamental part of each of the other disciplines and influences 
what they are. This is a very condensed treatment of learning organizations. 

Additional insight can be gained from the 9 Principles and 27 Lessons we developed in 
a short course: “Lessons Learned in Engineering” and a NASA CR report [NASA 
CR-201 1-216468] that are summarized below. 

I. System success depends on the creativity, judgment, and 
decision-making skills of the people 

1 . People Are the Prime Resource for Project Success 

2. People Skills are Mandatory for Achieving Successful Products 
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II. Space systems are challenging, high performance systems 

3. Demand for High Performance Leads to High Power Densities 

and High Sensitivities 

III. Everything acts as a system (whole) 

4. Systems and Technical Integration 

5. Risk Management 

6. All Design is a Paradox, a Balancing Act 

IV. The system is governed by the laws of physics 

7. Physics of the Problems Reigns Supreme 

8. Engineering is a Logical Thought Process 

9. Mathematics Is The Same! 

10. Fundamentals of Launch Vehicle Design 

V. Robust design is based on our understanding of sensitivities, 

uncertainties, and margins 

1 1 . Robustness 

12. Understanding Sensitivities and Uncertainties is Mandatory 

13. Margins Must Be Adequate 

VI. Project success is determined by life cycle considerations 

14. Design Space Constrained by Where You Are in the Life Cycle 

15. Concept Selection and Design Process 

16. Requirements Drive the Design 

17. Designing for the -ilities and Cost 

VII. Testing and verification have an essential role in development 

18. Hardware and Data Have the Answers 

19. Can Test Now or Will Test Later 

20. Independent Analysis, Test, and Design are Keys to Success 

21 . All Analyses and Tests are Limited 

22. Scaling is a Major Issue 

VIII. Anticipating and surfacing problems must be encouraged 

23. Must Hear and Understand All Technical and Programmatic Opinions 

24. There are No Small Changes! 

25. Expect the Unexpected 

IX. Leadership is the foundation 

26. Integrity 

27. Focus Beyond Yourself 
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Summary 

In summary, we have looked at the process whereby excellence in engineering can be 
achieved in a project and an organization. Directors of Engineering at Marshall Space Flight 
Center have added the core values that an organization must have in achieving the goal of 
successful products: 


Have Integrity 
Internalize Teamwork 
Be Safe 
Communicate 
Engage 

Make Yourself Better 
Leave It Better 
Do It Now 


The following figure (Figure 26) depicts the process of achieving excellence in 
engineering, starting with understanding root causes of problems, instituting the triangle of 
Technical Understanding and Excellence, Partnership with the Project, and Individual and 
Organizational Culture. Product success depends on the application of all the principles of 
engineering excellence. 


Summary to Success 


Product Success 


Root Causes of 
NASA Failmes Problems 

Shifting from engineering "hands on" and 
"excellence “to "i nsi ght/over si ghft " . Lack of 
ownership 


Lack of critical thinking. Over -reliance on procedures 
and computer codes 
Decentralization of authority. 

Organizational and technical complexity. 



Requires Integration 
Of All Principles 
Of Engineering Excellence 


Application of principles of engineering excellence ameliorates root causes 
and achieves product success 


Figure 26. Summary to Success 
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In closing there is an attitude that must permeate the organization and the individuals 
that can be classified as servanthood. The poem The Bridge Builder by Will Allen Dromgoole 
sums up this attitude. 


The Bridge Builder 

An old man, going a lone highway, 

Came, at the evening, cold and gray, 

To a chasm, vast, and deep, and wide, 

Through which was flowing a sullen tide. 

The old man crossed in the twilight dim; 

The sullen stream had no fears for him; 

But he turned, when safe on the other side, 

And built a bridge to span the tide. 

“Old man,” said a fellow pilgrim, near, 

“You are wasting your strength with building here; 
Your journey will end with the ending day; 

You never again must pass this way; 

You have crossed the chasm, deep, and wide- 
Why build you this bridge at evening tide?” 


The builder lifted his old gray head; 

“Good friend, in the path I have come,” he said, 
“There followeth after me today 
A youth, whose feet must pass this way. 

This chasm, that has been naught to me, 

To that fair-haired youth may a pitfall be. 

He, too, must cross in the twilight dim; 

Good friend, I am building the bridge for him.” 

- Will Allen Dromgoole 
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